Taming Gradient Variance in Federated Learning with Networked Control Variates

Federated learning, a decentralized approach to machine learning, faces significant challenges such as extensive communication overheads, slow convergence, and unstable improvements. These challenges primarily stem from the gradient variance due to heterogeneous client data distributions. To address this, we introduce a novel Networked Control Variates (FedNCV) framework for Federated Learning. We adopt the REINFORCE Leave-One-Out (RLOO) as a fundamental control variate unit in the FedNCV framework, implemented at both client and server levels. At the client level, the RLOO control variate is employed to optimize local gradient updates, mitigating the variance introduced by data samples. Once relayed to the server, the RLOO-based estimator further provides an unbiased and low-variance aggregated gradient, leading to robust global updates. This dual-side application is formalized as a linear combination of composite control variates. We provide a mathematical expression capturing this integration of double control variates within FedNCV and present three theoretical results with corresponding proofs. This unique dual structure equips FedNCV to address data heterogeneity and scalability issues, thus potentially paving the way for large-scale applications. Moreover, we tested FedNCV on six diverse datasets under a Dirichlet distribution with {\alpha} = 0.1, and benchmarked its performance against six SOTA methods, demonstrating its superiority.Comment: 14 page

AWTE-BERT:Attending to Wordpiece Tokenization Explicitly on BERT for Joint Intent Classification and SlotFilling

Intent classification and slot filling are two core tasks in natural language understanding (NLU). The interaction nature of the two tasks makes the joint models often outperform the single designs. One of the promising solutions, called BERT (Bidirectional Encoder Representations from Transformers), achieves the joint optimization of the two tasks. BERT adopts the wordpiece to tokenize each input token into multiple sub-tokens, which causes a mismatch between the tokens and the labels lengths. Previous methods utilize the hidden states corresponding to the first sub-token as input to the classifier, which limits performance improvement since some hidden semantic informations is discarded in the fine-tune process. To address this issue, we propose a novel joint model based on BERT, which explicitly models the multiple sub-tokens features after wordpiece tokenization, thereby generating the context features that contribute to slot filling. Specifically, we encode the hidden states corresponding to multiple sub-tokens into a context vector via the attention mechanism. Then, we feed each context vector into the slot filling encoder, which preserves the integrity of the sentence. Experimental results demonstrate that our proposed model achieves significant improvement on intent classification accuracy, slot filling F1, and sentence-level semantic frame accuracy on two public benchmark datasets. The F1 score of the slot filling in particular has been improved from 96.1 to 98.2 (2.1% absolute) on the ATIS dataset

Graph Learning and Its Applications: A Holistic Survey

Graph learning is a prevalent domain that endeavors to learn the intricate relationships among nodes and the topological structure of graphs. These relationships endow graphs with uniqueness compared to conventional tabular data, as nodes rely on non-Euclidean space and encompass rich information to exploit. Over the years, graph learning has transcended from graph theory to graph data mining. With the advent of representation learning, it has attained remarkable performance in diverse scenarios, including text, image, chemistry, and biology. Owing to its extensive application prospects, graph learning attracts copious attention from the academic community. Despite numerous works proposed to tackle different problems in graph learning, there is a demand to survey previous valuable works. While some researchers have perceived this phenomenon and accomplished impressive surveys on graph learning, they failed to connect related objectives, methods, and applications in a more coherent way. As a result, they did not encompass current ample scenarios and challenging problems due to the rapid expansion of graph learning. Different from previous surveys on graph learning, we provide a holistic review that analyzes current works from the perspective of graph structure, and discusses the latest applications, trends, and challenges in graph learning. Specifically, we commence by proposing a taxonomy from the perspective of the composition of graph data and then summarize the methods employed in graph learning. We then provide a detailed elucidation of mainstream applications. Finally, based on the current trend of techniques, we propose future directions.Comment: 20 pages, 7 figures, 3 table

Case report: EBV-positive epithelioid follicular dendritic cell sarcoma with CD30 expression: a highly challenging diagnosis

IntroductionFollicular dendritic cell sarcoma (FDCS) is a rare tumor entity with a wide range of anatomical sites and strong heterogeneity in morphology and immunohistochemistry, making it highly susceptible to misdiagnosis. There are two types of FDCS: conventional FDCS and EBV+ inflammatory FDCS. It is currently suggested that the former has nothing to do with EBV infection. Moreover, they have distinctively different clinicopathological characteristics.Case descriptionA 69-year-old male patient was admitted to our hospital after 4 months of progressive enlargement of the neck mass. Positron emission tomography/computed tomography (PET/CT) examination showed multiple enlarged lymph nodes in the body. After cervical lymph node excision and biopsy, it was found that the tumor cells were epithelioid and diffusely expressed EBER and CD30. It was initially diagnosed as poorly differentiated cancer and lymphoma. In subsequent differential diagnosis, we found that it strongly stained CD21 and CD23, which was approved the diagnosis of EBV+ FDCS.ConclusionEpithelioid FDCS is very rare. EBV-positive FDCS with abnormal expression of CD30 has not been reported. Whether EBV also plays an important role in conventional FDCS requires more cases to be verified. Our case provides valuable research clues for further understanding the pathological characteristics of this tumor entity

The Aroma Composition of Baby Ginger Paocai

The purpose of this study was to analyze the volatile compounds in baby ginger paocai and the fresh baby ginger and identify the key aroma components that contribute to the flavor of baby ginger paocai. A total of 86 volatile compounds from the two baby ginger samples were quantified; these compounds were extracted by headspace solid-phase microextraction (HS-SPME) and analyzed by gas chromatography–mass spectrometry (GC-MS). The aroma composition of baby ginger paocai was different from that of fresh baby ginger. Baby ginger paocai was characterized by the presence of aroma-active compounds which varied in concentration from 0.03 to 28.14%. Geranyl acetate was the aroma component with the highest relative content in baby ginger paocai. β-myrcene, eucalyptol, trans-β-ocimene, Z-ocimene, linalool, decanal, cis-citral, geraniol, geranyl acetate, curcumene, and β-bisabolene contributed to the overall aroma of the product of baby ginger paocai which had gone through a moderate fermentation process

Genetic diversity fuels gene discovery for tobacco and alcohol use

Tobacco and alcohol use are heritable behaviours associated with 15% and 5.3% of worldwide deaths, respectively, due largely to broad increased risk for disease and injury1,2,3,4. These substances are used across the globe, yet genome-wide association studies have focused largely on individuals of European ancestries5. Here we leveraged global genetic diversity across 3.4 million individuals from four major clines of global ancestry (approximately 21% non-European) to power the discovery and fine-mapping of genomic loci associated with tobacco and alcohol use, to inform function of these loci via ancestry-aware transcriptome-wide association studies, and to evaluate the genetic architecture and predictive power of polygenic risk within and across populations. We found that increases in sample size and genetic diversity improved locus identification and fine-mapping resolution, and that a large majority of the 3,823 associated variants (from 2,143 loci) showed consistent effect sizes across ancestry dimensions. However, polygenic risk scores developed in one ancestry performed poorly in others, highlighting the continued need to increase sample sizes of diverse ancestries to realize any potential benefit of polygenic prediction

Effect of Chitosan Coating with Different Molecular Weights on the Storage Quality of Postharvest Passion Fruit (Passiflora edulis Sims)

To study the preservation effect of chitosan coating with different molecular weights on postharvest passion fruit, the "Qinmi No.9" was coated with chitosan of molecular weights of 30, 50, 100, 150 and 200 kDa (1.5%, w/v) to determine the quality of passion fruit during storage. The results showed that chitosan coating with different molecular weights was able to delay the shrinkage and yellowing, reduce the weight loss rate and inhibit the decay of passion fruit. Moreover, chitosan with a larger molecular weight was more conducive to delaying the ripening and senescence of passion fruit, as well as reducing shrinkage, and decay. At the end of storage, the weight loss of fruits coated with 200 kDa chitosan was nearly 10% less than that coated with 30 kDa chitosan, and the fruits coated with 150 and 200 kDa chitosan did not decay. The lower molecular weight (30 and 50 kDa) and higher molecular weight (150 kDa) chitosan were more effective in inhibiting weight loss, total soluble solids and soluble sugar metabolism, and maintaining titratable acid, flavonoid and total phenol contents of fruit during storage. The chitosan with 150 kDa had the best effect in maintaining the vitamin C content, which was 1.12 times higher than the control group at the end of storage. In conclusion, chitosan with different molecular weights was effective to delay senescence, slow down water loss and shrink of passion fruit and maintain the quality, chitosan with 150 kDa was more suitable to maintain the quality of postharvest passion fruit

Impact of charged ionic species (NaCl and KCl) on the generation of color and volatile aroma compounds during caramelization

During the process of caramelization, volatile and non-volatile flavor and color are generated via the degradation of carbohydrates. This research investigated the effect of the types and concentrations of salts including NaCl and KCl on the generation of volatile aromas and color during caramelization. The solid phase microextraction-gas chromatograph-mass spectrometry (SPME-GC-MS) was used to measure the volatile compounds generated in caramelization. The results demonstrated that increasing content of salt (NaCl or KCl) could significantly improve (P<0.05) the generation of some essential volatile compounds during caramelization such as furfural, 5-methylfurfural, 5-hydroxymethyl-furfural (HMF), propionic acid and butyric acid. However, the ascending amount of salt (NaCl or KCl) had no significant impact (P<0.05) on the color generation of caramel. In conclusion, the usage of salt was beneficial to the generation of more aromatic compounds during caramelization

Discovery of potential biomarkers for osteoporosis using LC/GC−MS metabolomic methods

PurposeFor early diagnosis of osteoporosis (OP), plasma metabolomics of OP was studied by untargeted LC/GC−MS in a Chinese elderly population to find possible diagnostic biomarkers.MethodsA total of 379 Chinese community-dwelling older adults aged ≥65 years were recruited for this study. The BMD of the calcaneus was measured using quantitative ultrasound (QUS), and a T value ≤-2.5 was defined as OP. Twenty-nine men and 47 women with OP were screened, and 29 men and 36 women were matched according to age and BMI as normal controls using propensity matching. Plasma from these participants was first analyzed by untargeted LC/GC−MS, followed by FC and P values to screen for differential metabolites and heatmaps and box plots to differentiate metabolites between groups. Finally, metabolic pathway enrichment analysis of differential metabolites was performed based on KEGG, and pathways with P ≤ 0.05 were selected as enrichment pathways.ResultsWe screened metabolites with FC&gt;1.2 or FC&lt;1/1.2 and P&lt;0.05 and found 33 differential metabolites in elderly men and 30 differential metabolites in elderly women that could be potential biomarkers for OP. 2-Aminomuconic acid semialdehyde (AUC=0.72, 95% CI 0.582-0.857, P=0.004) is highly likely to be a biomarker for screening OP in older men. Tetradecanedioic acid (AUC=0.70, 95% CI 0.575-0.818, P=0.004) is highly likely to be a biomarker for screening OP in older women.ConclusionThese findings can be applied to clinical work through further validation studies. This study also shows that metabolomic analysis has great potential for application in the early diagnosis and recurrence monitoring of OP in elderly individuals